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Abstract: We study sparsity and spectral properties of dual frames of a given 
finite frame. We show that any finite frame has a dual with no more than n 2 
non-vanishing entries, where n denotes the ambient dimension, and that for most 
frames no sparser dual is possible. Moreover, we derive an expression for the 
exact sparsity level of the sparsest dual for any given finite frame using a general- 
ized notion of spark. We then study the spectral properties of dual frames in terms 
of singular values of the synthesis operator. We provide a complete characteriza- 
tion for which spectral patterns of dual frames are possible for a fixed frame. For 
many cases, we provide simple explicit constructions for dual frames with a given 
spectrum, in particular, if the constraint on the dual is that it be tight. 

1 Introduction 

Redundant representations of finite dimensional data using finite frames have received consider- 
able attention over the last years (see [10] for an extensive treatment including many references 
concerning various aspects). Finite frames are overcomplete systems in a finite dimensional 
Hilbert space. Similar to a basis expansion, frames can be used both to represent a signal in 
terms of its inner products with the frame vectors, the frame coefficients, and to reconstruct 
the signal from the frame coefficients. As for bases, the frames used for representation and 
reconstruction depend on each other, but are not, in general, the same. In contrast to basis ex- 
pansions, however, even once one of the two frames is fixed, the other, the dual frame, is not 
unique. Hence the quality of a method for redundant representation will depend on both the 
frame and the dual frame. There is, however, a standard choice for the dual frame, the so-called 
canonical dual, corresponding to the Moore-Penrose pseudo-inverse, whose optimality in many 
respects follows directly from its geometric interpretation. This is why most previous work con- 
cerning the design of finite frames has focused primarily on the design of single frames rather 
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than dual frame pairs, as it would be necessary to take into account all degrees of freedom in the 
problem. In particular, design according to spectral properties [4] and sparsity [6] has recently 
received attention. 

In this paper, we will take a different viewpoint; we study the set of dual frames for a given 
fixed finite frame. The guiding criteria, however, will again be sparsity and spectral properties. 
This is motivated by work in the recent papers [8, 16], which ask the following question: If 
the frame {0/}^ is given by the application at hand, e.g., by the way of measuring the data, 
which dual for the synthesis process is the best to choose? The precise answer to this question 
is, of course, dependent on the application, but universal desirable properties of the dual can, 
nonetheless, be recognized. Among such desirable properties are fast and stable reconstruction. 
Measures for the stability of the reconstructions resulting from a dual frame are extensively 
discussed in [8]. It is shown that the computational properties of the dual frames are directly 
linked to spectral properties of the dual frame matrix. In particular, the Frobenius norm and 
the spectral norm play an important role in this context. The analysis in [8] focuses on stability 
estimates when the frame coefficient vector is corrupted by random or uniform noise. The 
papers [13, 14, 17] discuss an alternative scenario of corruptions through erasures. The fact that 
the canonical dual is shown to be optimal in many cases also suggests connections to spectral 
properties, but we will not explore this connection further in this paper, leaving it to potential 
further work. 

A fast reconstruction process is closely linked to fast matrix- vector multiplication properties 
of the dual frame matrix. Matrices with these properties can be constructed in various ways. 
However, many such constructions yield matrices with a specific structure, often linked to the 
fast Fourier transform, which can be counteracting the constraint that the matrix should be a 
dual to the given frame. The criterion of sparsity, while also connected to fast matrix-vector 
multiplication properties, does not lead to such problems, but rather provides a way to ensure 
both duality to a given frame and a fast matrix- vector multiply. The first paper on the topic [16] 
constructs sparse duals of Gabor frames using i\ -minimization, and [8] also uses sparsity as a 
measure for computational efficiency. 

The focus of [8] has been on deriving criteria for comparing computational properties of 
dual frames and empirically balancing them to find a good dual frame. This paper is meant to 
be complementary to [8] as we analytically study the set of all dual frames of a given frame 
with respect to sparsity and spectral properties. We ask how sparse a dual frame can be and 
what we can say about the spectrum. 

Our paper is organized as follows. In Section 2 we present more formal definitions of frames 
and their duals as well as a review of results in frame theory, an overview of the notation we will 
be using, and some basic observations. In Section 3 we then consider the sparsity properties of 
the set of all dual frames. Finally, Section 4 focuses on the spectral picture of the set of all dual 
frames. 

2 Setup and basic observations 

2.1 Frames and their duals 

Throughout this paper, we are considering signals x in a ^-dimensional Hilbert space Jtf'n. We 
will often identify with K", where K denotes either K or C, using a fixed orthonormal basis 
of M'n. We write this canonical identification as = K n . Thus, W or will be the signal 
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space and K' n the coefficient space. Then the precise definitions of frames and their dual frames 
are the following. 

Definition 2.1. A collection of vectors 4> = C Jt? n is called a frame for if there are 

two constants < A < B such that 

m 

A N|2<£|(x,4> ; >| 2 < B\\x\\l, for all xeJ? n . (2.1) 

If the frame bounds A and B are equal, the frame (0;)^Li is called tight frame for ,Jtf? n . 

Let 4> = (^)^Li be a frame for Jt? n . Since we identify J%, = K'\ we also write for the 
coefficient (column) vector of and 4> = [(^"ij G K nxm as a n x m matrix, and we say that 
4> is a frame for K n . We write y for the jth row of 4>, and <J>^ = [0 J ]™ =1 G K^ -1 )*™ as 
a (n — 1) x m submatrix of 4> with the &th row deleted; the yth column of 4>( fc ) is denoted by 
■ G K' 7_1 . We also use the notation 4> = [<j>ij] or [<£]ij = <l>ij. For an index set / C { 1 , . . . , m}, 
we denote the restriction of 4> to these column vectors by 4>/ = 

Definition 2.2. Given a frame 4>, another frame *P = (y02=i G =^ is said to be a dual frame 
of 4> if the following reproducing formula holds: 

m 

* = £ (jc, 0/) y/i forallxGJ^. (2.2) 
(=l 

In matrix notation this definition reads 

= I n (2.3) 

or, equivalently, = /„, where /„ is the n x n identity matrix. So the set of all duals of 4> is 
the set of all left-inverses *F to 4>* (or the adjoints of all right-inverses to $). 

If m > n, then the frame (^O^Lj is redundant, i.e., it consists of more vectors than necessary 
for the spanning property. For these frames there exist infinitely many dual frames. 

Given a collection of m vectors 4> = in Jrf? n , the synthesis operator is defined as the 

mapping 

m 
!=1 

The adjoint of the synthesis operator is called the analysis operator and is given by 

F*:J£-MK m , x^((x,0,})7 =1 . 

In applications the analysis operation typically represents the way data is measured, whereas 
the synthesis procedure represents the reconstruction of the signal from the data. 

The identification M'n = K' 1 translates to identifying the synthesis mapping F with the matrix 
4> and F* with 4>*. The frame operator S : Jj?„ -> J4? n is defined as S = FF*, that is, 

m 

S(x) = £<x,fc)to (2.4) 
(=1 
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in matrix form the definition reads S = 4>4>* . It is easy to see that 4> is a frame if and only if the 
frame operator defined by (2.4) is positive definite. In this case, the following reconstruction 
formula holds 

m 

x = ^ (x,S <j>i)<j)ii for all* G J^ n , (2.5) 

hence, ¥ := (S _1 0;)£L i is a dual frame of <I> in the sense of Definition 2.2, it is called the 
canonical dual frame. The canonical dual *F has frame bounds 1 /B and 1 /A, where A and B 
are frame bounds of <J». In terms of matrices, *F is the Moore-Penrose pseudo-inverse & = 
cJ>*(c£kJ>*) 1 of the analysis matrix 4>*. 

2.2 Sparsity and spectrum 

The measure we use for sparsity is the number of non-vanishing entries of the dual frame which 
we denote, with a slight abuse of notation, by || • | j o - This convention as well as the associated 
name £o-norm (although it is not a norm) is common practice for vectors and relates to the fact 
that it can be interpreted as the limit of £ p -quasinorms, p — > + . In our paper, the term norm 
will also be used in a sense that includes the £o- norm - 

As for the spectral properties of a frame <I>, we will mostly work with singular values (Oi)'- =l 
of the synthesis matrix <J». While one often studies the eigenvalues (A;)" =1 of the frame operator 

= 4>4>* in related cases, the advantages of the singular value approach is that one then 
has the frame directly accessible, whereby the construction of dual frames becomes, if not 
straightforward, then at least tractable. Obviously, the values are related by A, = (cr,) 2 for all 
i = 1, .. . ,n. 

2.3 The set of all duals 

By the classical result in [15], all duals to a frame 4> can be expressed as 

(2.6) 

i=l 

where r\i G for i = 1 , . . . , m is arbitrary; in matrix form it reads: 

where E is a matrix representation of an arbitrary linear mapping from K m to J4? n . The canonical 
dual appearing in these formulas can be replaced by any other dual, that is, if *F is any dual, 
then all duals are given by 

^ + £(7™ -<£>*¥). (2.7) 

The matrix 4>* V I' is the so-called cross Gramian of the dual frame pair. Here E : K m — > 
can be an arbitrary linear mapping, but different choices of E can yield the same dual, so there 
are less degrees of freedoms than entries in E. In fact, if one considers <I> as given and *F 
as our unknown, equation (2.3) corresponds to solving n independent linear systems, each of 
which consists of n equations in m variables. Hence, the set of all duals *F to a frame <J> is an 
n{m — n) -dimensional affine subspace of Mat(K, nxm). 

A natural parametrization of this space is obtained using the singular value decomposition. 
Let 4> = UL&V* be a full SVD of 4>, i.e., U G K" x " and V G K mxm are unitary and L<p G R nxm 
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is a diagonal matrix whose entries are the singular values \/B = 0\ > o 2 > ■ ■ ■ > o n = \fk > 
of 4> as its diagonal entries Oj > in a non-increasing order. Let *P be a frame and define 
Mvp := f/*W e K nxm , where U and V are the right and left singular vectors of 4>. Then *P 
factors as *F = t/MipV*. By 4> V P* = /„, we then see that 



/„ = U*I n U = U*&¥*U = 
Therefore, ^ is a dual frame of <J> precisely when 

where *F = UM^V* . The solutions to (2.8) are given by 



£<j,Mvj/. 



(2.8) 



I/O! 

I/CT2 








*1,2 
*2,1 *2,2 











l/<J n S n \ S n ,2 



51, r 

52, r 



(2.9) 



.. ,r = m — n. Note that the canonical dual frame is 
, n and k = I,. ...m — n. 



where Sj^ E K for i = 1 , . . . , n and k = 1 , 
obtained by taking s^i = for all i = 1 , . . 

3 Sparsity of duals 

The goal of this section is to find the sparsest dual frames of a given frame <J» e 
question can be formulated as the minimization problem 



This 



min W^l 



s.t. 



10 a - u x — ^n- (3.1) 

Definition 3.1. We call the solutions of (3.1) sparsest duals or optimal sparse duals. 

Let us consider a small example illustrating that solutions to this minimization problem are 
not unique. 

Example 1. We consider duals of the frame 4> = (0i , fc, <fe) in IR 2 defined by 



4> 



1 -1 
1 2 



(3.2) 



All duals *P are easily found by Gauss- Jordan elimination to be given by 



hence the sparsest duals are seen to be 
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and ^ 



h,t 2 e 
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(3.3) 



(3.4) 



which correspond to the parameter choices t 2 = —1 and t\ = — 2, 0, 1 , respectively. These are the 
only three duals with ||*P|| = 3. According to the measures derived in [8], the non-degenerate 
dual m 2 without zero vectors is preferred. This reflects the fact that it is the only one of the 
three duals for which one is not discarding frame coefficients in the reconstruction procedure. 
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Solving (3. 1) of sizes much larger than the 2x3 example in Example 1 is not feasible since 
the problem is NP-complete. Hence to find sparse duals, one has to settle for approximate 
solutions of (3.1), e.g., by convex relaxation or greedy strategies. For such approaches we refer 
to [8, 16]. In the following subsections we will focus on analyzing the possible values of H^Hq, 
rather than on how one finds minimizers. An exception is Section 3.3, where finding the sparsest 
duals is tractable due to the specific structure of the frame. 

3.1 Upper and lower bounds for the minimal sparsity 

In this section we investigate the possible sparsity levels in the set of all dual frames. In other 
words, we consider the possible objective function values of the minimization problem (3.1). 
We start with a trivial upper bound for the sparsity level of sparse duals. 

Lemma 3.1. Suppose 4> is a frame for K n . Then there exists a dual frame *P of<t> with 

H^llo^ 2 - (3-5) 

Proof. Choose J C {1,2, ... ,m} such that |7| = n and the vectors {fa}ieJ are linearly indepen- 
dent. This is always possible, as otherwise, the fa do not span the space and hence do not form a 
frame. Let { V/j/e/ be the unique (bi-orthogonal) dual of {fa}iej, and set = for i ^ J. Then 
obviously ||*F|| < n 2 holds. □ 

Similarly to Lemma 3.1 we see that the corresponding trivial lower bound on the dual frame 
sparsity is n, that is, any dual frame satisfies ||*P||o > n. However, we can give a much better 
lower bound in terms of the spark of the 4^'s. Here the spark of a matrix 4> G Mat(K,« x m) 
is defined as the smallest number of linear dependent columns of <J» and denoted by spark (<3>). 
In case 4> is an invertible n x n matrix, one sets spark (4>) = n + 1. 

Theorem 3.2. Suppose 4> is a frame for K n . Then any dual frame *P of<i> satisfies 

n 

||¥|| > £ spark (4>W). (3.6) 

Proof. Fix j = I,... ,n. By equation (2.3), the jth row of V P, that is, \\fi , must be orthogonal to 
§ k for k = 1 , . . . , j — 1 , j + 1 , . . . , n. Therefore, we have that 

n _! = <^V')* = [Aesuppv* V 7 'L P p^ (3-7) 

where 0„_i is the (n-l)xl zero column vector. Hence, the || ty^'llo columns : j G supp i/a- 7 } 
of <t>(^ are linearly dependent, which implies that || V J ||o > spark (4>( ; )) and thus the result. □ 

It is easy to see that the lower bound on the support given in Theorem 3.2 is not always 
sharp. Indeed, if 4> has two linearly dependent columns, one has spark (4>^) = 2 for all j, 
but there is not necessarily a dual with only two entries in each row. In order to predict the 
maximally possible sparsity, one needs a more refined notion that excludes this type of effects 
arising from linearly dependent columns. More precisely, denote by sparky (4>) the smallest 
number of linearly dependent columns in 4>( J ) such that the corresponding columns in 4> are 
linear independent. Obviously, we have that sparky (4>) > spark (4>^) for every j = l,...,n. 
Using this refined notation of spark we can improve the bound obtained in Theorem 3.2 for 
optimal sparse duals. Even more, the following result exactly describes the sparsity level of the 
sparsest dual. 
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Proposition 3.3. Suppose <J> z'^ a frame for K". Then the sparsest dual frame *P of<& satisfies 

||Y|| =£ spark,- (<!>). (3.8) 

Proof. Let ^ be a sparsest dual of 4>. Then (0j[^) fe6su ^ must be linearly dependent in order 

to allow for <t>^ = w _i. However, the frame vectors (<fc)&esuppiW must be linearly inde- 

pendent, as otherwise one of these columns can be expressed through the others, which would 
allow for the construction of with suppy/ 7 C suppi//- 7 such that ^(w J ) = ej. This in turn 
would imply that the frame, whose synthesis matrix has rows yf \jf- ' , y/ J , ty/ J+1 , . . . , \j/ n , 
is also a dual frame of <J», so *F is not the sparsest dual. Thus | supp i// J | > sparky (4>), which 
implies that no dual frame can have less than E'j=i sparky (4>) non-zero entries. 

To show the existence of dual frames with this sparsity, let S be a set of size sparky (4>) 
such that ((j>k)keS ls a set °f independent columns of <J» such that the corresponding columns 
of 4>( J ) are linearly dependent. That is, there exist (^k)keS sucn tnat JlkeS^k(^^)k = 0, but 
(LkeS kk<l>k) j = a^0. Hence the vector given by 

satisfies 4>(i// J )* = e,-, and the matrix *F with rows yields a dual with a frame matrix support 
size of Ey=i sparky (4>), as desired. □ 

3.2 Frames with the lower sparsity bound n 2 

In Lemma 3.1 we saw that it is always possible to find a dual frames with sparsity level n 2 . In 
this section we will show that for a large class of frames this is actually the best, or rather the 
sparsest, one can achieve. 

Recall that anxm matrix is said to be in general position if any sub-collection of n (column) 
vectors is linear independent, that is, if any n x n submatrix is invertible. The following result 
states that if the projection of <I> onto any of the n coordinates hyperplanes of dimension n — 1 
is in general position, then the frame has maximal dual sparsity level. This observation follows 
from Theorem 3.2. 

Theorem 3.4. Suppose 4> is a frame for K w such that the submatrix <&U) is in general position 
for every j = 1 , . . . , n. Then any dual frame of<t> satisfies 

||¥|| >n 2 . (3.10) 

In particular, the sparsest dual satisfies ||*P|| = n 

Proof. Fix j = 1, . . . and let us consider c = (ci) 1 ^ =1 = (v^)* as a m x 1 column vector to be 
chosen. Then 



£c ; 0. (j) =()„_!. (3.11) 
1=1 
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Since <3>( fe ) is in general position, any collection of n — 1 vectors from (0; is linearly 
independent. Hence, equation (3.11) can only be satisfied if c has more than n— 1 nonzero 
entries, that is, if \{i : c, ^ 0}| > n. Hence, each row of *P must have at least n nonzero entries, 
which yields a total of n 2 nonzero entries. □ 

We illustrate this result with a number of examples of frames which are well-known to be in 
general position, and which thus do not allow for dual frames with less than n 2 non-vanishing 
entries. 

Example 2. For any n, m G N with m > n, let a,- > 0, i = 1 , . . . , m with a,- ^ aj for all i ^ j, and 
let bj > 0, j = 1, . . . ,n with 7^ Z?;, j 7^ i. We then define the generalized Vandermonde frames 
as 



4> 



b\ /?2 /?» 
^ 1 ^ 1 ••• (X j 



(3.12) 



It is not difficult to see that the submatrix 4>( J ) is in general position for every j = l,...,m. 
This yields a deterministic construction for a frame 4> 6 K" xm such that any dual frame *F of 4> 
satisfies 

ll^llo^" 2 - 

Remark 1. We remark that any minor of a generalized Vandermonde frames 4> is non-zero, 
which, in particular, implies that the submatrices <t>^ are in general position for every j = 
1, . . . ,m. Neither of these properties hold, in general, for standard Vandermonde matrices. A 
standard Vandermonde frame 4> is obtained as the transpose of the matrix in (3.12) with bj = 
j—1 for j = 1 , . . . , n and a ( - e K for i = 1 , . . . , m. Such a standard Vandermonde frame is again in 
general position, see also [1]. To see that this is not necessarily the case for all <&v\ take n = 3 
and let a i() = — 1 and a;, = 1 for some ?'o and i\ G { 1 , . . . , m}. Then the submatrix <J>( 2 ) restricted 
to the indices {z'o, z'i } is a 2-by-2 matrix of all ones, hence <J>( 2 ) cannot be not in general position. 

The generalized Vandemonde frames introduced above are almost never used in applications 
due to the fact that for m 3> n they are necessarily poorly conditioned, both in the traditional 
sense and in the frame theory sense as introduced in [8]. The following example yields well- 
conditioned, deterministic frames for which no dual frame with less than n 2 entries can exist 
either. 

Example 3. Let n E N, and let m be prime. Let 4> be a partial FFT matrix or a harmonic tight 
frame matrix of size nxm. Then any <t>(ri is in general position, as the determinant of any 
(n — 1) x (n — 1) submatrix of 4> is non-zero. This is a consequence of Chebotarev theorem 
about roots of unity stating that any minor of an m x m DFT matrix is non-zero whenever m is 
prime [19-21]. Thus again, the sparsest dual frame *F of 4> satisfies 

ll^llo = " 2 - 

Example 4. Let n e N be prime and let m = n 2 . It was shown in [11] that for almost every 
a G C", the Gabor frame generated by a has the property that any minor of its frame matrix is 
non-zero. We conclude as before that the sparsest dual frame satisfies 

ll¥L = ft 2 . 
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In fact, the property of having a sparsest dual with sparsity n 2 is a generic property. To 
precisely formulate this observation, let JP(n,m) be the set of all frames of m vectors in Jrf? n , let 
jV{n, m) be the set of all frames of m vectors in whose sparsest dual has n 2 non-zero entries, 
and let £?(n, m) be the the set of all frames 4> which satisfy spark (<J>( J )) = n for all j = 1 , . . . , n. 
We remark that the following result can be obtained using techniques from algebraic geometry 
[1]; we include an elementary proof for illustrative purposes. 

Lemma 3.5. Suppose m>n. Then the following assertions hold: 

(i) The set &(n,rri) is open and dense in K" xm , that is, £P(n,m) c is closed and nowhere 
dense. 

(ii) The set &{n,m) c is of {induced Euclidean) measure zero. 

Proof, (i) Let 4>o E K" xm be a given. Pick <J»i E ^(n,m), e.g., a generalized Vandermonde 
frame. Define 

% = t<&\ + (1 -t)®o teK. 

We claim that only finitely many of {4> f : t E K} are not in ^(n,m). To see this we introduce 
the following polynomial in t: 

j=ues 

where S is the collection of all subsets of { 1 , . . . , m} of size n — 1 and [Uj^/GKHlxf*-!) is 
the matrix 4> f without row j and restricted to the columns in the index set /. Clearly, \S\ = ( n ™i) , 
and P is therefore of degree « 2 („™ 1 ) • By definition, we see that 

P(f)=0 

<^> det( [<£,]/) = for some I E S and j = 1 , . . . , n 
<^ spark (<&j . ) < n for some j = 1 , . . . , n 

Therefore, P(t) ^ if and only if 4> f E g?(n,m). Since P(l) ^ 0, the polynomial P(t) is not 
the zero polynomial hence it has finitely many zeros, to be precise, it has at most n 2 ( n ™j) zeros. 
Therefore, we can choose \t\ arbitrarily small such that 4>/ E &(n,m). Consequently, 4> is 
arbitrarily close to a frame in ^(n, m). 

Part (ii) follows in a similar setup as in the prove of (i) using an integration argument. □ 

By Theorem 3.2 we see that &(n,m) C ,yV(n,m) and thus JP(n,m) \jV(n,m) C JP(n,m) \ 
£P(n,m). Hence, the following result immediately follows from Lemma 3.5. 

Theorem 3.6. Any frame in JF(n,m) is arbitrarily close to a frame in jV{n,m). Moreover, the 
set &(n,m)\ jV{ri, m) is of measure zero. 

Another consequence of Lemma 3.5 is that for many randomly generated frames, the spars- 
est dual has sparsity level H^Hq = n . As an example, this holds when the entries of 4> are 
drawn independently at random from a standard normal distribution. Also frames obtained by 
a small Gaussian random perturbations of a given frame will have this property. 
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3.3 Sparse duals of sparse frames 

We have seen in the previous sections that, while the sparsity of the sparsest dual of a frame 
can lie between n and n 1 , generically, its value is n 2 . One can, however, by imposing structure, 
obtain classes of frames that allow for considerably sparser duals. One example is a class of 
very sparse frames, the so-called spectral tetris frames. 

The spectral tetris algorithm is constructs unit norm frames of m vectors in W 11 with pre- 
scribed eigenvalues of the frame operator. It was developed in [5] and extended in [2]. The 
construction works for any sequence of eigenvalues (Aj) . =1 that satisfy Aj > 2 for all j and for 

which m := Y!)=\ Aj is an integer. 

We will need to introduce some notation that we will use throughout this section. For a 
sequence (Aj) . =1 , we define and m,- as follows. Set Icq = and mo = and recursively 

choose ki = inf{N 3 k > : m ; - := Ej=i Aj G N}. Naturally, the largest value jl that can be 
chosen for i is such that = m and k^ = n. Furthermore, let K := {ki : i = 0, 1, . . . , jl}. 

Definition 3.2. For a sequence (Aj) .j satisfying the condition m := £y=i Aj G N, the Spectral 
Tetris frame STF(n,m, {Aj}) G R nxm is given by 



1 a\ 



m\ times 



"1 



where, for j K, aj :- 



1 «2 ^2 

—^2 1 



OT2 times 

2 and & j : 



1 a n _i a„_i 

ra„ times 



1 — y, and r, G [0, 1) and m ; - G No are defined by 



X j = mj + r j 



h = (l-rj-il+mj + rj, 



when j—leK, 
otherwise. 



(3.13) 
(3.14) 



If j G K, the 2 x 2-block matrix Bj 



is omitted. 



,n, and m := Ej=i Aj G N. Lef 



and 



Proposition 3.7. Suppose (Aj) . =1 satisfies Aj > 2, j = 1,.. 
4> := STF(n,m, (Aj) . =1 ). Define the set 7,7 C {l,...,n} fry 

/ = {0<7</i-l:*;+i =*;+!} 

r jo 

7 = < jo G + l,fc ;+ i) : / = 0,...,/i anJ £ ^/ 

and Ze£ £ := 2/1 + |/| — |/|, where jl is the maximal index i as above. Then the sparsest dual *P 
satisfies 

\\^\\ = k+2(n-k). 





Jo-1 




( 


I h 






J=ki+\ _ 
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Proof. Let («/)/= l denote the standard orthonormal basis of E n . If ej G 4>, then spark ; (4>) = 1 
since contains a zero column. On the other hand, if ej 4>, then spark (4>) = 2. Let 



k=\{j: ej e<t>}\. 
By Proposition 3.3, it then follows directly that the sparsest dual satisfies 

n 

||¥|| = £ sparky (<£) = k+2{n-k). 



(3.15) 



To complete the proof, we need show that k always equals k = 2\x + \ J\ — i.e., that there are 
exactly 2/x + |7| — |/| rows of 4> containing a 1. 

For that, recall that each kj corresponds to a 2 x 2-block matrix Bj = ^ _ J b . being omitted 
at the transition from one row to the next. Hence the fcyfh column is e m ., the (kj + l)st column 
is e m + i; each of the /i values of kj yields two rows that contain an ej. if j G (KU (K+ 1)) fl 
{1 , . . . ,n}, then ej G 4>. Correcting for the double counting resulting from the case kj+\ =kj + l, 
there are 2/i — |/| values of £ such that eg G 4> resulting from this phenomenon. 

In addition, there can be e( G <J> that do not correspond to such a transition, i.e., that appear 
as the j'oth column of the frame matrix, where jo is such that k t ■ + 1 < Jo < h+l for some i. For 
such a jo we calculate 



m jo + r jo 



%o- 2 + 0'o-l 

%o - 2 + %o-l - 2 + r jo-2 - m ;o-i 

(*7o + A Jo- 1 ) - ( 4 + m Jo- 1 ) - Oo-2 
70 / 70 _ 1 

•••= I A 7 -(2(jo-1)+ £ 



m 7 



By definition the j'oth row contains a 1 if and only if m J() > 1. Furthermore, it can easily be 
shown, e.g., by induction, that 



Jo-l 


70-1 


2C/o ; 


L m J = 


I % 


7= 


ki+l 




Therefore, the j'oth row of 4> contains a 1, if and only if 


7o / 


70-1 




E h-[ 


E h 


+ 2^ > 1 


j=ki+l \ 


J=*H-1 . 





+ 2. 



This is just the defining condition of J, so one has an additional number of |7| vectors ei G 4>, 
which proves the theorem. □ 

Remark 2. Since ji > 1, we see that k > 2. Thus the sparsest dual of a Spectral Tetris frame 
satisfies n < || v I'|| = 2n — fc< 2/7 — 2. Furthermore, if Ay > 3 for all /= 1, we see that 
||*P||o = n since k = n. 
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Remark 3. Recall that the spectral tetris algorithm is only guaranteed to work for redundancies 
larger than two, see Definition 3.2. In [12] the proof of Proposition 3.7 is used to characterize 
when the spectral tetris construction works for tight frames with redundancies below two. In 
fact, a similar argument works for general frames without a tightness condition. 

Remark 4. A construction scheme for optimal sparse duals of Spectral Tetris frames directly 
follows from the proof of Proposition 3.7. The resulting dual will consist only of 1-sparse 
vectors and 2-sparse vectors. 

4 Spectral properties of duals 

In this section we answer the question of which possible spectra the set of all dual frames 
admits. A special case of this question has already been answered in [18], namely the question, 
which frames admit tight duals. In linear algebra terms this boils down to the question whether 
a full-rank matrix <J» has a right inverse with condition number one. The first step towards a 
more general characterization will be a more quantitative version of their result. In particular, 
we study which frame bounds tight duals can attain for a given frame. Furthermore, our proof 
provides an explicit construction procedure for these duals. The understanding developed in the 
proof will turn out useful in some more general cases. 

It turns out that a frame always has a tight dual if the redundancy is two or larger. If the 
redundancy is less than two, it will only be possible under certain assumptions on the singular 
values of 4>. Note that in [18], the result is formulated in terms of the eigenvalues of the frame 
operator, so even the existence part of the following result is formulated slightly different from 
the result in [18]. 

Theorem 4.1. Let n,meN. Suppose 4> is a frame for W with m frame vectors and lower frame 
bound A. Then the following assertions hold: 

(i) Ifm > 2n, then for every c > there exists a tight dual frame \F with frame bound c. 

(ii) Ifm = 2n—l, then there exists a tight dual frame *P; the only possible frame bound is 

(Hi) Suppose m < 2n — 1. Then there exists a tight dual frame *P if and only if the smallest 
2n — m£ {2, . . . , n} singular values of<i> are equal. 

Proof. Let 4> = UL<t>V* be a full SVD of 4>, and let *F be an arbitrary dual frame. Following 
Section 2.3, we factor the dual frame as *F = UM^V*, where M^> is given as in (2.9) with 
$i k G K for i = 1 , . . . , n and k= 1, . . . , r = m — n. For *F to be tight, we need to choose sn such 
that the rows of Mvp are orthogonal and have equal norm. This follows from the fact that *F is 
row orthogonal if and only if My is row orthogonal. 

As the diagonal block of My is well-understood, the duality and tightness constraints trans- 
late to conditions for the inner products of the Si = (s^i, . . • € K r , i = 1, . . . ,n. Indeed, *P 
is a tight dual frame with frame bound c if and only if, for all 1 < i < n, one has 



and, for all i ^ j = 1, . . . , n, one has (st, Sj) = 0. 

Now assume that o n — <7„_i = • ■ • = o p+ \ < o p for some p <n. The case of distinct singular 
values corresponds to p = n — 1. As <J p +i < Oi for all 1 < / < p, (4.1) implies that all Si for 



(4.1) 
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i = 1, ... ,p must be nonzero vectors even if s p+ i,. . . ,s n are all zero. Furthermore, (4.1) also 
determines the norms of s\ , . . . , s p as a function of \\s p+ i || = • • • = \\s n \\ . The second condition 
implies that if s n ^ 0, the sequence (f/)y=i is orthogonal, else the sequence (si)f =1 . 

If r > n, that is, if m > In, then any choice of s n allows for an orthogonal system with com- 
patible norms, so tight dual frames with any frame bound above exist and can be efficiently 
constructed. If r < n, then no n vectors can form an orthogonal system, one needs to have s n = 
and hence also Sj = for all j > p. So no frame bound other than is possible. The remaining 
vectors {j_/}^ =1 are all non-zero, so they must form an orthogonal system. For r > n — p + 1 , this 
is possible, and again a solution satisfying the norm constraints can be efficiently constructed. 
For r < n — p, no such system exists, hence there cannot be a tight dual. 

This completes the proof. □ 

We remark that for tight frames <I>, hence p = 0, one has the following two cases: If m < In, 
then 4> has only A l <t> as tight dual frame. If m > In, then 4> has infinitely many tight dual 
frames, in particular, 4> has a C-tight dual frame for any C > A . 

We will now derive general conditions on which spectral patterns (now possibly consisting 
of more than one point) can be achieved by a dual frame of a given frame. The reason that, 
in the general framework, such an analysis is harder than in the context of tight duals is that 
in that case, the frame operator is a multiple of the identity, hence diagonal in any basis. This 
no longer holds true if we drop the tightness assumption, so when the orthogonality argument 
of Theorem 4.1 fails, one cannot conclude that there is no dual with a given spectral pattern. 
However, the orthogonality approach allows to choose a subset of the singular values of the dual 
frame freely. In particular, if the redundancy of the frame ^> is larger than 2, it follows that for all 
spectral patterns satisfying a set of lower bounds, which we will later show to be necessary (see 
Theorem 4.6), a dual with that spectrum can be found using a constructive procedure analogous 
to the proof of Theorem 4. 1 . 

Theorem 4.2. Let n,m e N, and let 4> be a frame for W 1 with m frame vectors and singular 
values (Oi)" =l . Suppose that r < m — n and that I r C {1, . .. ,n} with \I r \ = r. Then, for any 
sequence (qi)iei r satisfying qt > l/oifor all i € I r , there exists a dual frame *F o/4> such that 
{qi}iei r is contained in the spectrum of^. Furthermore, it can be found constructively using a 
sequence of orthogonalization procedures. 

Proof. The proof is just a slight modification of the proof of Theorem 4.1. Again, we choose 
( s i)iei r to be orthogonal and the remaining Si's to be the zero vector. The non-zero s ; vectors are 
scaled to satisfy 

a 2 - — + IU-II 2 

Hi — ^2 ^ H A 'll2' 

where i G I r . Hence, by this procedure we obtain a dual frame with spectrum {qi}iei r U {o^ 1 }igi r - 

□ 

As a corollary we obtain that using the same simple constructive procedure, one can find 
dual frames with any frame bound that is possible. 

Corollary 4.3. Let 4> be a redundant frame for W with singular values ((J,)" =1 . Fix an upper 
frame bound satisfying > \ and a lower frame bound — > A* > \, where we use the 

G n °m-n+l CT 1 
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convention = °° if m> 2n. Then a dual frame mofQ with these frame bounds can be 

found constructively using a sequence of orthogonalization procedures. 

To obtain a complete characterization on the possible spectra for dual frames, we will need 
the following two results by Thompson [23] on the interlacing properties of the spectrum of sub- 
matrices. To simplify notation in the following proof, we use a simpler formulation than in [23], 
which is technically speaking a special case of the setup given in [23], but up to permutation 
captures the results in complete generality. 

For our setup, let A E ~K nxm with singular values d\ > . . . 0C min („ m ), where m,n E N. Fur- 
thermore, for q,£ E N, £ > q, let P q : K. e — > K q denote the restriction on the q first entries. Of 
course, the P q depends not only on q, but also on the input dimension £, but as this can usually 
be inferred from the context, we suppress this dependence to simplify notation. Note that P q P q 
is the projection onto span{e ; - :/=!,...,<?}, i.e., 



xt if i < q, 
else, 



and PqPq is the identity on K q . Furthermore, we take p,q E N such that p < n and q < m. 

Theorem 4.4 ( [23, Theorem 1]). Suppose j8i > • • • > /3 m i n ( p ^) are the singular values of B E 
KP xq given by B = P p AP*. Then 

CCi > pi for l<i< min {p, q} , (4.2) 

A > ^i+(m-p)+(n- q ) far i < min{p + q - m,p + q - n} . (4.3) 

Theorem 4.5 ( [23, Theorem 2]). Let j8i > • • • > P m i n ( p . q ) satisfy (4.2) and (4.3). Then there 
exist unitary matrices U E K nxn and V E K mxm such that P p UAV*P* E K pxq has singular 

values (An 1 ™ 1 - 

These two results now allow for a complete characterization of the possible spectra of dual 
frames. 

Theorem 4.6. Let n,m E N, and set r = m — n. Let 4> be a frame for W 1 with singular values 
(oi')F=i- Suppose is any dual frame with singular values (a ; >i ')" =1 (also arranged in a non- 
increasing order). Then the following inequalities hold: 

~^—<o^ fori=l,...,r, (4.4) 

Gn-i+l 

' <C7*< 1 fori = r+l,...,n. (4.5) 



Gn-i+l Gn-i+r+1 

Furthermore, for every sequence (0 , * p )" =1 which satisfies (4.4) and (4.5), there is a dual *F o/"4> 
with singular values (a ; ^)" =1 . 

Proof. To show the necessity of the conditions, we will apply Theorem 4.4 for p = q = n. Note 
that this entails that P p = I n . Let 4> = be a full SVD of 4>. We factor dual frames of 4> as 

^ = UMyV*, where My is given as in (2.9) with Si± eKfor i = 1,.. . ,n and k= l,...,r = m—n. 
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We let Mq, denote the case where we choose s^k = for all i, k; in this case = UM~V* is the 
canonical dual. Note that MyP* £ K nxn has the same spectrum as MyP*P n = Mq,, namely 
singular values ( 1 / a„_, + i )" =1 . Furthermore, and My have the same singular values (o~ ( - )" =1 . 
A direct application of Theorem 4.4 for A = My and B = MyP* 6K" X " now shows (4.4) and 
(4.5). 

For the existence, we need to show that there are unitary matrices U £ K" x ' 1 , V £ K mxm 
such that for the diagonal matrix Ly £ R" xra with diagonal entries (o^- )" =1 one has 

<f>VZyU* =I n . (4.6) 

For this we will apply Theorem 4.5 for A = 4>, (ce)" =1 = (o - ,-)? j, and again p = q = n. Let a 
sequence (o^)f =1 which satisfies (4.4) and (4.5) be given. Note that for our choice of parame- 
ters, this is equivalent to having (A')f=i = (V°n > • • • > ^/ G T) sat isfy (4.2) and (4.3). Hence, by 
Theorem 4.5, there exist unitary matrices U\ £ K nx " and Vi £ K mxm such that [/i$V*P n * and 
hence ^V^P* have singular values ((o , ; vp ) _1 )"_ 1 (in a non-decreasing order). By yet another 
SVD, there exist unitary matrices [/, Vi £ K nxn such that U*<t>VlP*V2 is an n x n diagonal ma- 
trix with entries ((cr ; >i ') _1 )" =1 (note that this time, the variant of the SVD with singular values 
in a non-decreasing order is used). Thus, we have 

U*<t>V?PZV£PnLy = I n (4.7) 
and, noting that (I m - P*P n )Zq, = 0, this leads to 

QVf (I m - P,:P n + F*V}P n )?XgU* = I n . (4.8) 
A direct calculation using the fact that P n P* = I n shows that 

V = Vt {I m - P* n P n + P^Pn) eK mxm (4.9) 

is unitary, so we obtain (4.6) as desired. □ 

The inequalities (4.4) and (4.5), written in terms of the singular values (<7 ( - )" =1 of the canon- 
ical dual frame *F := S have the following simple form: 

mm 

<5, < G{ for i = 1 , . . . , r, 

5/ m TO 

Ci < Gi < Gi-r for i = r+ 1, . . . ,n. 

We remark that the first part of Theorem 4.6 also follows from r applications of [7, Theorem 
7.3.9] on the matrix My defined in (2.9) or from the well-known interlacing inequalities for 
Hermitian matrices by Weyl. 

Also note that the proof of Theorem 4.5 and hence the proof of Theorem 4.6 involves exis- 
tence results for unitary matrices from [22] and is, consequently, less intuitive and constructive 
than our above proof for the special case in Theorem 4.2. For this reason, we decided to include 
Theorem 4.2, even though it directly follows from the general existence result of Theorem 4.6. 

In terms of eigenvalues of frame operators, Theorem 4.6 states that the spectra in the set of 
all duals exhaust the set A C W defined by 

A = |(Ai, . . .,Xn) £ E n : X? < X t < Xf_ r for all / = 1,. . . ,n} , 
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where A* = l/X„_ i+l is the z'th eigenvalue of the canonical dual frame operator; we again 

use the convention that A ( - = °° for i < 0. By considering the trace of M^Mq,, we see that 
the canonical dual frame is the unique dual frame that minimizes the inequalities in A. Any 
other spectrum in A will not be associated with a unique dual frame, in particular, if 5 ; - = 
(sil, . . . ,Si >r ) 7^ in M>p for some i = l,...,n, then replacing 5,- by zst for any \z\ = 1 will 
yield a dual frame with unchanged spectrum. The frame operator and the canonical dual 
frame operator are connected by = Sq, + C, where C = W*W, and W G K rxn is arbitrary. 
Hence, in linear algebra terms, Theorem 4.6 tells us that for a given positive definite, Hermitian 

matrix Sq, with spectrum (A ; - )" =1 and for a given sequence (Aj)" =1 G A, there exists a positive 
semi-definite, Hermitian matrix C of rank at most min{m — n,n} such that the spectrum of 
:= Sq, + C consists of (A,-)" =1 . 

For a better understanding of the more general framework where Theorem 4.2 does not yield 
a complete characterization of the possible spectral patterns, we will continue by an extensive 
discussion of the example of a frame of three vectors in IR 2 . 

Example 5. Suppose 4> is a frame in IR 2 with 3 frame vectors and frame bounds < A* < 5*, 
and let 4> = be the SVD of 4>. Then all dual frames are given as \F = UMyV*, where 



1/cti si 

1 /O2 52 



for 5i , S2 G K. Since the frame operator of the dual frame is given by 5>p 
we can find the eigenvalues of 5>p by considering eigenvalues of 



These are given by 



^1,2 



S:=MyMy 



1 1 

-trS±-R, 

2 2 



l/crf + sj s\s 2 

5152 + 



where 2? = W (tr5) 2 - 4det5. 



One easily sees that trS monotonically grows as a function of s\ + 5 2 , whereas for fixed trS, the 
term R grows as a function of s\ — 5 2 . This exactly yields the two degress of freedom predicted 
by the existence part of Theorem 4.6. A straightforward calculation shows that R + (5 2 + s 



> 



> 0, hence we see that 



h > 



and 



> A 2 > 



T 2 ) 



(4.10) 



which is also the conclusion of the necessity part of Theorem 4.6. We remark that the two 
eigenvalues depend only on quadratic terms of the form s\ and 5 2 . Therefore, if s\ and 52 
are non-zero, then the choices (±^,±$2) yield four different dual frames having the same 
eigenvalues. In this case the level sets of Ai as a function of (51,52) are origin-centered ellipses 
with major and minor axes in the 51 and 52 direction, respectively. Moreover, the semi-major 
axis is always greater than sq := (<J 2 2 — cf 2 ) 1//2 . The level sets of A2 are origin-centered, 
East-West opening hyperbolas with semi-major axes greater than so. In Figure 1 the possible 
eigenvalues of the dual frame operator of the frame $ defined by 



4> = 



50 



90 
120 



-12 
9 



-16 
6 



(4.11) 
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are shown as a function of the two parameters s\ and S2', Figure lb shows the level sets and the 
four intersection points (±*i , ±52) for each allowed spectrum in the interior of A. Note that the 
singular values are 0\ =3 and 02 = 1/2, hence 5* = 9 and A* =1/4. 




(a) Graphs of X\ and A2 



(b) Level curves of X\ and A2 



Figure 1 : The lower and upper frame bounds of dual frames *P (to <J> defined in (4.1 1)) as a function of 
s\ and S2- The two graphs in (a) meet at si = ±\/35/3 and S2 = which correspond to tight dual frames. 



When the difference between the singular values of *F goes to zero, the ellipses degenerate 
to a line segment (or even to a point if 0\ = 02). The limiting case corresponds to tight dual 
frames so Theorem 4.1(ii) applies, and we are forced to set $2 = to achieve row orthogonality 
of Mip. We then need to pick si such that the two row norms of My are equal, thus 

which shows that the above lower bound for the semi-major axis is sharp. 
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